Model Hyper Parameters

Training dates 2016-09-01 to 2019-12-30

Transformations

Channel Diminishing Returns Shape Decay rate
TV log 0.98
Digital 0.9 0.3
SEM Brand 0.2 0.4
SEM Non-Brand 0.9 0.4
Social 0.9 0.6
App 0.9 0.93

Model Performance

\(R^2\) = 0.8907251

MAPE = 4.19%

MAPE (Test) = 4.40%

RMSE = 2,389

RMSE (Test) = 2,430

Estimate t value Pr(>|t|)
(Intercept) -25233.68 * -1.441 0.150
tv 9.654 3.168 0.002
semBR 155.388 2.401 0.017
semNB 0.043 2.884 0.004
app 0.024 5.076 0.000
social 0.056 3.943 0.000
NA NA NA NA
email_marketing 0 * -0.901 0.368
email_operational 0 4.546 0.000
email_savedsearch 0 * -0.249 0.803
SEO_bigcity 39028.293 2.156 0.031
SEO_hdp -14710.895 -2.533 0.011
push_android 0 * -1.528 0.127
push_ios 0 2.016 0.044
nonbrand_queries 0.3 15.498 0.000

*Not significant effect

Lift Analysis

Lift is the expected contribution of the channel to the overall prediction. “If we removed the channel, what percent of total Submits would we lose?”
2017 2018 2019
(Intercept) -55% -57% -57%
tv 12% 11% 11%
semBR 3% 4% 4%
semNB 2% 3% 3%
app 4% 5% 8%
social 3% 4% 6%
digital NA% NA% NA%
email_marketing 0% 0% 0%
email_operational 5% 3% 4%
email_savedsearch 0% 0% 0%
SEO_bigcity 85% 88% 86%
SEO_hdp -29% -29% -28%
push_android -1% -1% -1%
push_ios 1% 1% 2%
nonbrand_queries 72% 73% 77%

CPA Analysis

date tv semBR semNB app social digital
2016-09-01 $15.41 $5.03 $41.39 $7.25 $18.54 Inf
2017-01-01 $22.46 $5.38 $40.58 $7.59 $20.03 Inf
2018-01-01 $25.16 $5.81 $41.16 $7.99 $20.59 Inf
2019-01-01 $20.93 $6.42 $41.07 $8.06 $21.20 Inf

Error Plot

Variable Analysis

Variable Correlation

Media Marginal Plots

Variable plots

Owned

## [1] "email_marketing"   "email_operational" "email_savedsearch"
## [4] "push_android"      "push_ios"

Apendix

Seasonal Variable

Estimate t value Pr(>|t|)
christmas -9,569 -9.842 0.000
easter -4,688 -4.503 0.000
fathers -2,657 -2.542 0.011
halloween -2,054 * -1.614 0.107
independence -9,477 -6.384 0.000
labor -2,591 -3.439 0.001
martin 1,130 * 0.781 0.435
memorial -3,824 -4.306 0.000
mothers -3,278 -3.124 0.002
new -6,527 -6.267 0.000
presidents 2,924 2.016 0.044
super -801 * -0.551 0.582
thanksgiving -4,569 -6.381 0.000
valentines -4,661 -3.238 0.001
year2017 280 * 0.658 0.511
year2018 -1,459 -2.390 0.017
year2019 -5,967 -6.930 0.000
month2 2,671 6.449 0.000
month3 1,744 4.221 0.000
month4 1,310 2.818 0.005
month5 -1,712 -3.602 0.000
month6 -1,779 -3.523 0.000
month7 -203 * -0.394 0.694
month8 -638 * -1.431 0.153
month9 -2,640 -6.608 0.000
month10 -3,495 -8.047 0.000
month11 -4,969 -9.611 0.000
month12 -7,271 -15.097 0.000
wdayMonday 3,184 7.809 0.000
wdaySaturday -468 * -1.326 0.185
wdaySunday -2,333 -5.165 0.000
wdayThursday 1,487 4.253 0.000
wdayTuesday 2,526 6.633 0.000
wdayWednesday 2,354 6.491 0.000